Error detection and correction in signal transmission by use of convolution codes



J. L. MASSEY Sept. 17, 1968 ERROR DETECTIGN ANDA CORRECTION IN SIGNAL TRANSMISSION BY USE OF CONVOLUTION CODES 18 Sheets-Sheet l SSEY ERROR DETECTION AND C ECTION IN SIGNAL TRANSMISSION BY USE OF CONVOLUTION CODES Filed Aug. 2, 1963 18 Sheets-Sheet 2 Sept. 17, 1968 J. MA 3,402,393

F l G, 2

|74 36 |76 ALARM THRESHOLD DEVICE f4() 78 u u u a n l Qc) 73 |72 I T 1 A 3" 40'5Il 7" g l I BEEQEEQBQE QE QEEE E J 2o 22 l 24 2e Y j 32 SO 1 174 A. 54 392es|70 F IG 3 THRESHOLD DE\/|CE 3640,;778

27 T- |7 DECODER KEY- QD AND GATE NOR" uNlT DELAY QQ M00- Two ADDER N U) '7e E C YHE "5 ERROR E lOl Ll I V '//A A A A A A ALARM 21m |04 |05 |03 v l F l G. 4 102 |05 A A A A E Sept. 17, 1968 J. L. MASSEY 3,402,393

ERROR DETECTICN AND CORRECTION IN SIGNAL TRANSMISSION BY USE OF `CONVOLUTION CODES Filed Aug. 2, 1963 18 Sheets-Sheet 5 F,|G 6 J ABO)\/(ABO)V(AOO)V BOO) BLOCK NAME A TYPICAL OIROUxT n l FOR NONE'.l FOR ZERO @z AND GATE ORl GATE 5,5

Sept. 17, 1968 J. l.. MAssEY 3,402,393

ERROR DETECTICN AND CORRECTION IN SIGNAL TRANSMISSION BY USE OF' CONVOLUTION CODES Filed Aug. 2, 19635 18 Sheets-Sheet 4 SOLUTION SIGNAL CLOCK 1 EVENT COUNTER E 4 JT 4o? Csar/AQP2 EVENT MATRIX 1 /417 4:13 REC|RCULAT|ON C@ A V IN PUT GAIESA/L T SCALE OF QW) Two 40o lTRRESHOLQ COUNTER F I G. 8 SOLUTION s|GNAl 1 Sept. 17, 1968 1. L. MAssr-:Y 3,402,393

ERROR DETECTION AND CORRECTION IN SIGNAL TRANSMISSION BY USE OF CONVOLUTION CODES Filed Aug. 2, 1963 18 Sheets-Sheet 5 TulLu-r iOr ilr I2r '3f i4r i5r i6r i7r `ir m-Zl' im-r'mr D0 so: POr +5 QO'\ DI 8| P|r +2 QI go D2 s2= P2r+2 g2 gl go D3 Ss Pr +2 Qa 02 9| go D4 S4: P4r +2 94 93 92 9| 9o D5 S5 F5r +2 Q5 Q4 Q3 Q2 a; QN D6 Se p6r +2 Q6 Q5 Q4 Q5 Q2 9| Q0 D7 S7 F'7r +2 Q7 Q6 Q5 Q4 93 02 0| D8 Se P8r +2 ge Q7 0e Q5 94 93 J2 9| om2 Sm 2=Pm-2r+2gm.2' a8 Q7 a6 o5 o4 93192@ Q0 Dm" Sm-FPm-'r JfS-fJnn-:Qrfwa' Qa Q7 Qs Q5 04 93592 9| 1V0\ Dm Sm :Pmr +9m Qm-lgme' Qa Q7 9e Q5 Q4Q3 Q2 9| Q0 F IG. 9

i i i i Y `Ollr 2r 3|- 4r 5r 'Dr E j@ So Plr E I DSI P2r E 3r 3 Fr+| l-@E-Swsq Ftr E I @Y ADDS3 D5 o5 Q4 Q3 o2 o, Q0 ORDINARY F I G. IO s, v* 5 ARITHMIETIC IO o eo' D COMPARE I' CD# l' TO er 5 THRESHOLD 0D VALUE Sept. 17, 1968 J. L. MAssEY 3,402,393

ERROR DETECTION AND CORRECTION IN SIGNAL TRANSMISSION BY USE OF CONVOLUTION CODES Filed Aug. 2, 1963 18 Sheets-Sheet 6 SET OF ORTHOGONAL v SUBSOLUTION 0r 'lr i2 i3r 54|. l5r SIGNALS Por 2 Q9@ SD or 'Dlr 2 P2r D3 PBH E l LJ [55 53,-@ P4r 2 Y SI+S4-iO P5rl+ 2 l (D Q) S5ior l 5 a5 o4 Q3 LQ0 |D4||D3| ADD ORDINARY F IG. H ADDITION 5 COMPARE TO ion THREsHoLD VALUE i i 2r ir 4f i5IV 'er *Tr i'sr ier lor in,r i12,

Pl E

Pn E

P12 2 g|2 F IG. I3

Sept, 17, `'l-"S' J. L. MAssEv 3,402,393

ERROR DETECTION AND CORRECTION IN SIGNAL TRANSMISSION BY USE OF CONVOLUTION CODES Filed Aug. 2. 1963 18 Sheets-Sheet 7 WWWWWWWWWWWWW 3,402,393 ssIoN J. l.. MASSEY Sept. 17, 1968 ERROR DETECTIGN AND CO Filed Aug. 2. 1963 18 ASheets-Sheet 8 2 a gn-4 2 go gf 2 2 Pn+ gn g 0 ...m Illy 4 o n.0 3 ...w \n90 3 O O 303| 3% l nQOnQI O O. 5 ...|52 3. .l2 n n|n2 0 Q G. 9h Q0 Q u. 3|3235 3%! In. ODO! H0200.: a. G. Q. M Q Q u. a. 4 0 3 2 3 5 3 4 3 n .Mv nOO n |n02 n03n04 Q Q Q Q Q Q Q u. a. E E E E hD O O 0 3 3 s 3 n n n 2 2 3 4 P Ps P4 Pn Po P P P s s 8 l l s l O O .u` 3 2 3 3 4 n HOO n 2 n 3T 4 S Q18 S S S S S.

sept. 17, 196s Filed Aug. 2, 1963 ERROR DETECTION AND CORRECTION IN SIGNAL TRANSM BY USE OF CONVOLUTION CODES 'J. L. MASSEY ISSION 18 Sheets-Sheet 9 2 2 a 854-85 466MB ADD COMPARE Sept. 17, 1968 1 L, MASSEY 3,402,393

ERROR DETECTION AND CORRECTION IN SIGNAL TRANSMISSION,

BY USE OF CONVOLUTION CODES Filed Aug. 2, 1963 18 Sheets-Sheet lO .I I .I K K K lor IIr Imr Ior O lr 0 |mr O SIOHI: PIDKIIFE gfJKaLII ZIH) r I) I) I I +I +I S562 F+g(lw)gw) vZKIUZ'Q) IKO-III (.I) I+II I II (+I) I+II i I+II SmTPmFI'EQrI 9m@ "'goKa- Zro ZmI 2go I Sgm a+ E gg@ 2go SIIIIOI PrOI+ E QIIOI gag() ZInOI ZInOI In@ (no) I@ InoI Inc) 50) @0I (no) SIn z Pmr+ gIII Qm-I` go Zrn Zm--I Zo F IG. I7

SION

18 Sheets-Sheet 1l Sept 17, 1968 J. l.. MAssEY ERROR DETECTIGN AND coRREcTloN 1N SIGNAL TRANSMIS BY USE 0F coNvoLUTIoN CODES Filed Aug. 2, 1965 ma@ L Sept. 17, 1968 J. l.. MAssEY 3,402,393

ERROR DETECTICN AND CORRECTION IN SIGNAL TRANSMISSION BY USE OF CONVOLUTION CODES Filed Aug. 2, 1963 18 Sheets-Sheet l2 llr lar |K Ir KI,

r z K l K F G. I9

Sept. 17, 1968 .1. l.. MASSEY 3,402,393

ERROR DETECTIGN AND CORRECTION IN SIGNAL TRANSMISSION BY USE OF CONVOLUTION CODES Filed Aug. 2, 1963 18 Sheets-Sheet l5 Om u Sept. 17, 1968 Filed Aug. 2, 1963 T I M E i lr 2r r J. ERROR DETECTIGN AND COR l.. MASSEY 3,

RECTION IN SIGNAL TRANSMISSI BY USE OF CONVOLUTION CODES 18 Sheets-Sheet 14 FIG. 22

Sept. 17, 1968 Filed Aug. 2, 1963 J MAssEY BY USE OF CONVOLUTION CODES SSION 18 Sheets-Sheet l5 i|r i2, i3, 54 AOD 4'5/ I34, +2 P S5 COMPARE 422 a pSr 2. l l S5 AOD Per +2 l l L41@ R Se COMPARE ADD 402 COMPARE l:Y l G. 23 ilr el,

:QQ-, l

i2r e2* i3 e lf r 3 vl b2* '3e K l G- 24 in 2r 3f i4' *S5 AOO p5r E E S7 COMPARE P6, 2 l s6 ADD 43d P7r f E 0 l l S7 COMPARE 43o/ADO 4152J im ,t COMPARE I4 4 if 466 '2 55982 @f g4 13 i [es (e3 (e2+e3) (63+e4) lr /LA [el L. MASSEY ERROR DETEOTION AND CORRECTION 1N SIGNAL TRANSMI SSION 18 Sheets-Sheet 1 G BY USE OF CGNVOLUTION CODES 8 6 w 9 1 2, 7, .0 1.. u u A t P e l s m..

IIr

l D l O O l O l O l O O I I I l l I O l O l l l l O O l l l O l l O O l l l l O l l O O l O E E 2 E E E E E 2 E f Y r. .l Y r Y f f f 7 mm H W. M ...m

F l O25 l O Smm s 4I Sl -IY Sl ZI.' Sl

MoJ

Sept- 17, 1968 1. MASSEY ERROR DETECTIGN AND CORRECTION IN SIGNAL TRANSMISSION BY USE OF CONVOLUTION CODES 18 Sheets-Sheet 17 L: 5f, of a @a S f LN. :la mi mi Lma l *Etf *Avmc Lmm; Lgt; E

@N .o u

Lm .Etwa

my www@ mwa www@ www www w. www www@ mwamwam MW NW wm www@ wams mw.. w@ y mi fn. i@ .Mmmm ,mma lmof 5 E sa Lof i@ ma LNE 5 @a i@ LQ@ mi L f ma 3,402.1,393 SSION Sept. 17, 1968 1. l.. MASSE-:Y

ERF-.OR DETECTGN AND CORRECTION IN SIGNAL TRANSMI BY USE OF CONVOLUTION CODES 1963 18 Sheets-Sheet 18 United States Patent O "ice 3,402,393 ERROR DETECTION AND CORRECTION IN SIG- A TSRANSMISSION BY USE OF CON VOLUTION O E James Lee Massey, South Bend, Ind., assignor to `Codex Corporation, Cambridge, Mass., a corporation of Delaware Continuation-impart of application Ser. No. 212,312, July 25, 1962. This application Aug. 2, 1963, Ser. No. 299,534

12 Claims. (Cl. 340-1725) ABSTRACT F THE DISCLOSURE Digit-al error correcting coding techniques involving the sensing of conditions under which the decoder is operating. In one aspect a monitoring device counts the ones in the decoding decision and in certain parity checks corrected by those decisions. In another aspect the code is supplemented such that one actual decoding error` can create a number of apparent errors or ones, which ena-bles early detection of conditions beyond the capability of the decoder. In another aspect fades are detected in combination with a decoding logic capable of acting upon the whole set or any group of nonfaded elements in the decoding set.

The present invention relates to apparatus for processing signal infromation, and, more particularly, to the correcting and/or detecting of signal errors or other changes, 4as produced in transmission. This application is a continuation in part of U.S. application Ser. No. 212,312, led July 25, 1962, now issued as U.S. Patent 3,303,333, and by this reference incorporated herein.

In signal-transmission systems involving the encoding of signal information, as in binary bits representing 0 or 1 symbols, noise and simil-ar factors, at play during the signal transmission process, may change the signal erroneously to cause the reception of a 1 or a 0 when the actual transmitted signals were 0 or l bits, respectively. It is to the solution of the problem of detecting such errors and preferably correcting the same that the present invention is primarily directed; it being an object of the invention to provide new and improved apparatus for such signal processing.

A further object is to provide a new and improved apparatus for detecting (and correcting, in important instances) errors caused by noise and similar factors in successive transmitted information signals.

Another object is to provide novel decoding apparatus of more general utility, as well.

Still another object is to provide a novel apparatus for majority decoding of coded signals, with particular, though not exclusive, emphasis upon convolutional codes.

An additional object is to provide -an improved majority decoding apparatus or the like with a novel weighting system,

A further object, still, is to provide a new and improved decoding apparatus embodying a threshold-operated decoder for group codes.

Other and further objects will be explained hereinafter and will be more particularly pointed out in connection with the appended claims.

In summary the invention provides novel coding apparatus employing digital error correcting coding techniques involving the sensing of conditions under which the decoder is operating. In one aspect a monitoring device counts the ones in the decoding decision and in certain parity checks corrected by those decisions. In another aspect the code is supplemented such that one actual decoding error can create ra number of apparent errors 3,402,393 Patented Sept. 17, 1968 or ones, which enables early detection of conditions beyond the capability of the decoder. In another aspect fades are detected in combination with a decoding logic capable of acting upon the whole set or any group of nonfaded elements in the decoding set.

The invention will now be described in connection with the accompanying drawing, FIG. 1 of which is a block diagram of a decoder apparatus described in my U.S. Patent 3,303,333.

FIG. 2 is a block diagram of means according to the invention for counting errors to monitor the operation of the decoder;

FIG. 3 is a block diagram of means for randomly supplementing the signals in the decoder according to the invention for highly sensitive monitoring of the decoder operation;

FIG. 4 is a block diagram of a decoder that operates on the binary erasure channel according to the invention;

FIG. 4a is a block diagram of modications according to the invention made to the embodiment of FIG. 12 for the case where nonerased signals have a small probability of being incorrect;

FIG. 5 is a circuit diagram of an analogue threshold device;

FIG. `6 is a -circuit diagram of 4a digital threshold device;

FIG. 7 is a circuit diagram of another digital threshold device;

FIG. 8 is a circuit diagram of a digital threshold device that employs a recirculation technique;

FIG. 9 is a general parity check diagram for rate 1/2 convolutional codes;

FIG. 10 is a parity check diagram of the components -and connections of the type I decoding apparatus described above in connection with FIG. 1;

FIG. 11 is a parity check diagram of the components and conections of the type II decoding apparatus described in my U.S. Patent 3,303,333 in connection with FIG. 4;

FIG. 12 is a parity check diagram of the components and connection of a decoder for a rate 1/2 constraint length 44 convolutional code;

FIG, 13 is a parity check diagram of the components vand connections of a decoder for a self-orthogonal convolutional code;

FIG. 14 is a general parity check diagram for rate 1/110 convolutional codes;

FIG. 15 is a parity check diagram of a decoder for a rate 1/3 constraint length 24 convolutional code;

FIG. 16 is 4a parity check diagram of a decoder for a rate 2/3 convolutional code;

FIG. 17 is a general parity check diagram for rate Ico/1z0 convolutional codes;

FIG. 18 is a block diagram of a forward-'backward decoder laccording to the invention;

FIG. 19 is a general parity check diagram for block codes;

FIG. 20 is a block diagram of the components and connections of a type I cyclic sequential decoder for a (7, 3) cyclic block code;

FIG. 21 is a block diagram of the components and connections of a type II sequential cyclic decoder for a (7, 3) cyclic block code;

FIG. 22 is a parity check diagram of the components and connections -of a sequential cyclic decoder for a Bose- Chaudri (15, 7) cyclic code;

FIG. 23 is a parity check diagram of the components and connections of a nonsequential decoder for block Codes;

FIG. 24 is a parity check diagram of the components and connections of a nonsequential decoder for block codes that enables step-wise orthogonalized decoding solutions;

FIG. 25 is a parity check diagram of the components and connections of a type I sequential decoder for the Bose-Chaudri (l5, 5) cyclic code which uses step-wise orthogonalization;

FIGS. 26-28, are circuit diagrams of the components and connections of a type II sequential decoder for the Bose-Chaudri (15, 5) cyclic code; and

FIG. 29 is a parity check diagram of the components and connections of a step-wise orthogonalized decoder for Reed-Muller codes.

The structure and operation of FIG. 1 are fully described in my U.S. Patent 3,303,333 to which reference is made. Excerpts from that patent follow:

At the input conductor 1 of the encoder portion 2 of the apparatus of FIG. l, successive signals representing symbols of information or data are applied, say one unit every two intervals of a predetermined reference clock period or frequency, i0, i2, i4, i6, i3, im, etc. lf this information were transmitted without encoding, some of the information symbols might become changed, as by noise or other factors in the system, and there would be no way of detecting or correcting those changes or errors. The encoder 2 is employed to supply redundancy to the information sequence; that is, to insert or intcrpose between the successive information signals, additional signal symbols which are linear functions of the information and which enable the detection aud/or correction of such errors, as later explained.

Let it be assumed, for purposes of illustration, that the information symbols are binary signal digits or bits; i.e., a l or a 0 bit. This may be encoded in a conventional convolution code, as suggested by P. Elias in an article entitled Coding for Noisy Channels, Institute of Radio Engineers Convention Record, Part IV, 1955, pp. 3744, or in other codes, also. For purposes of an example `only the invention is explained as applied to a systematic code having a rate 1/2 (every other signal a redundancy) and a constraint length of l2 (6 information signals within the encoder at any one time and a redundancy signal being generated each time the information signals are shifted one position).

In FIG. 1, the encoder 2 comprises a chain of five serially connected stages of shift register or time-delay 3, 5, 7, 9 and 11. At even instants of time, the symbols i0, i2, i4, i6, etc., are fed along conductor 13 to an adder or gate circuit 15 for passing the same to the encoder output 17.

Redundancy signals are computed by a further adder 19 into which signals are fed directly from the input conductor 1 by way of conductor g1', from the output of stage 7 (representing a 6-unit delay) by way of conductor gq; from the output of stage 9 (representing an S-unit delay) by conductor gg; and from the output of stage 11 (representing a l0-unit delay), by means of conductor gu. Conductors g3' and g5 (from the respective stages 3 and 5) ,may also be provided.

Thus, with an input comprising successive bits introduced two units of time apart, the input signal may be represented by the expression:

meaning, bit i2 is delayed two units; bit i4, four units; and so on. The encoder code or generator sequence may be of many different types, but the preferred convolutional code g is as follows:

gzll-giDl-l-gsDS-lgliD11 (2) meaning, that if i0 is l, and all other information signal symbols are 0, the encoder output will be unity at the initial time, and will have value g1 after one unit of time; g3 after three units of time; and so on. For a more general sequence, the encoder output t will be the product of the input (Equation 1) and the generator or code sequence (Equation 2), namely,

where l is the entire digital message stream, I is the entire digital stream of information components and g is the generator sequence. With systematic codes, that is, codes such as the example above in which the generator sequence commences with the integer l, each information component i0, i2, ii, etc., appears by itself in the message stream at some unit of time as well as becoming a term in a multiplicity of redundancy components in the rnessage. With such codes the information components i0, i2, i4, etc., and the redundancy components p1 (equaling glio), P3 (equalng gaio-biglie), P5 (equing gsiol-azl gli), etc., together comprise the digital message components of the stream t, each g1, g3, g5, etc., in the binary case can take the value of either l or 0, the particular gs that are ls depending on the specific code chosen. In this mathematical treatment, it is to be understood that the operations of multiplication and addition are carried out as in binary-number logic, such that l l=l, 1 0=0, 0 l=0 and 0 0=0; and 1+l=0, 0+0=0 and l-i-O or O-l-l=1 (so-called modulo-two arithmetic).

These encoded signals are shown transmitted in any desired manner to a receiving-and-decoding apparatus 4, schematically shown as transmitted along the path 17, with noise or other interfering factors electing possible changes in the transmitted sequence. At the even time-unit intervals corresponding to the original input signal-in formation bits i0, i2, i4, etc., there will be received, at the receiver decoder 4, corresponding signals iol., gl., 11h.' etc., some of which may be spuriously changed from l to 0, -or vice versa, by the effects of noise or other iniluences. These error effects may be indicated by symbols e0, e2, e4, etc., representing noise-signal influences, referred to herein as error components, which may be unity in the case of causing a change in the transmitted bit, or zero if no change has occurred. At the odd time units the redundancy signals p11., p31., p51., etc., are received, affected by noise components el, e3, e5, etc.

The rst function of the decoder 4 is to separate the received even-time information symbols for, 1'2., igt, etc. with their possible errors e0, e2, e4, etc.) from the supplemental or redundancy signals therebetween; in this case, the signals at the odd time-unit intervals. This is effected by an alternating or switching gate 6, operating at the frequency of the reference clock, alternately to feed successive received signals along paths 8 and 10 for even and odd time-unit intervals, respectively.

The even-time information symbol signals are preferably fed into a substantial replica of the original chain of shiftregister encoding circuit 3-57-911-1921, represented by the same symbols with prime notations 3-5-79 11'-19*21. Conductors g1" through gn" correspond to conductors 1f-gu of the encoder 2, with conductors g3 and g5" considered as disconnected or open-circuited for the illustrated example.

In the absence of any error E, the output of the single delay stage 21 will clearly be equal to the supplemental or redundancy signals at the odd time-unit intervals, referred to as simulated redundancy signals. This output is fed along conductor 12 to an adder 14 into which the odd time-unit interval redundancy signals from the message are directly fed by conductor 10. Thus, the output of adder 14 at 16 is always zero if no error components in the transmitted redundancy and information symbol signals r have occurred. This output at 16 may be represented by the product of the error components at the even time-unit intervals and the generator or code sequence (Equation 2) plus the entire noise sequence at the odd time-unit intervals, since any output at 16 can only be caused by the noise signals. The error signal components S of the output may be shown to comprise the following groups at the odd-unit instants of time: 

